New Advances on Bayesian Ying-Yang Learning System With Kullback and Non-Kullback Separation Functionals

In this paper we extend Bayesian Kullback YING YANG BKYY learning into a much broader Bayesian Ying Yang BYY learning System via using di erent sep aration functionals instead of using only Kullback Diver gence and elaborate the power of BYY learning as a gen eral learning theory for parameter learning scale selection structure evaluation regularization and sampling design with its relations to several existing learning methods and its developments in the past years brie y summarized Then we present several new results on BYY learning First im proved criteria are proposed for selecting number of den sities on nite mixture and gaussian mixtures for select ing number of clusters in MSE clustering and for selecting subspace dimension in PCA related methods Second im proved criteria are proposed for selecting number of expert nets in mixture of experts and its alternative model and se lecting number of basis functions in RBF nets Third three categories of Non Kullback separation functionals namely Convex divergence Lp divergence and Decorrelation index are suggested for BYY learning as alternatives for those learning models based on Kullback divergence with some interesting properties discussed As examples the EM al gorithms for nite mixture mixture of experts and its alter native model are derived with Convex divergence BYY Learning System and Theory